Stochastic Reachability: From Markov Chains to Stochastic Hybrid Systems

Stochastic hybrid systems represent established classes of realistic models of hybrid dynamics subject to random perturbations, autonomous uncontrollable transitions, nondeterminism or uncertainty. Stochastic reachability analysis is a key factor in the verification and deployment of stochastic hybrid systems. In this paper, we tackle the stochastic reachability problem from a purely probabilistic perspective. Using the connection between stochastic reachability and optimal stopping, the reach probabilities are estimated using only the probabilistic parameters (transition probabilities, infinitesimal generator) of stochastic hybrid systems. The technique is illustrated first for Markov chains, then for other more complex Markov models.